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Abstract 

Teams  are  increasingly  important  as  organizations  reengineer  to  meet  information  age  conditions 
and  objectives.  This  paper  examines  how  participant  ratings  of  command  and  control  planning 
and  observer  assessments  of  teamwork  were  related  in  a  series  of  futuristic  missions  conducted 
by  the  Mounted  Maneuver  Battlespace  Lab  at  Fort  Knox,  KY.  The  exploratory  results  indicate 
that  planning  and  teamwork  may  be  closely  related,  particularly  in  the  reengineered  command 
and  control  organizations  envisioned  in  the  2010-1-  timeframe. 

1.  Introduction 

Teams  are  increasingly  important  in  the  effective  functioning  of  information  age  organizations 
(Baker  &  Salas,  1997).  In  private  industry,  project  teams  with  members  throughout  the 
organization  are  often  formed  for  product  development  (e.g.,  user-centered  design/engineering). 
In  the  military,  peacekeeping  operations  may  require  team  members  from  combat  arms,  engineer 
and  medical  units,  as  well  as  non-government  organizations  such  as  the  Red  Cross  to  work 
together  to  successfully  accomplish  the  mission. 

As  the  importance  of  teams  and  team  behaviors  increases  more  research  is  focusing  on  the 
identification  and  determination  of  behavioral  factors  contributing  to  effective  teamwork.  As 
Ilgen  (1999)  points  out,  one  research  method  involves  examining  behaviors  that  distinguish 
effective  teams  from  less  effective  teams.  This  method  has  been  used  extensively  by  Navy 
researchers  (e.g.,  Oser  et  al,  1989).  Cannon-Bowers  et  al,  (1995)  provide  a  substantial  list  of 
behaviors  that  may  help  differentiate  effective  and  ineffective  teams.  These  behaviors  include 
adaptability,  shared  situational  awareness,  performance  monitoring  and  feedback, 
leadership/team  management,  interpersonal  relations,  coordination,  communication,  and 
decision-making . 

An  obvious  question,  however,  is  how  do  effective  teams  develop  these  teamwork  behaviors? 
One  possible  answer  is  anticipation  of  team  members'  needs.  Effective  teams  are  able  to  "push" 
information  and  action  to  team  members  before  it  is  needed  versus  team  members  having  to 
"pull"  information  and  actions  from  each  other  (Entin  &  Serfaty,  1999).  Underlying  the  notion 
of  anticipation  is  the  idea  of  a  shared  mental  model  of  the  team  and  the  task.  That  is,  because 
members  understand  the  team  task,  and  each  other's  task  roles  and  functions,  effective  teams  are 
able  to  anticipate  what  information  and  action  other  team  members  need. 

There  are  many  methods  by  which  team  members  can  acquire  anticipation  skills  and  shared 
mental  models.  These  include  cross  training  on  others'  jobs  (Cannon-Bowers  et  al.,  1998),  direct 
training  of  team  mental  models  (Duncan  et  al.,  1996),  and  training  feedback  on  teamwork 
behaviors  (Stout  et  al.,  1997). 
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Team  planning  may  be  another  important  method  for  learning  to  anticipate  other  team  members' 
needs.  Team  planning  usually  includes  discussion  and  rehearsal  of  team  tasks  prior  to 
performance.  Moreover,  team  planning  is  often  grounded  by  a  specified  task  context  and 
deadline  that  may  help  focus  team  members  anticipation  of  own  and  others  needs.  For  example, 
Stout  et  al.,  (1999)  examined  the  effects  of  team  planning  on  shared  mental  models,  proactive 
providing  of  information,  and  performance  errors  in  a  low  fidelity  simulation  called  Gunship. 
Results  showed  better  planning  teams  had  more  similar  mental  models,  proactively  provided 
more  information,  and  made  fewer  errors  under  high  workload.  These  results  make  sense,  in  that 
planning  should  involve  discussing  the  mission,  what  is  likely  to  happen,  what  different  team 
members  will  do  and  what  information  they  will  need  under  various  circumstances.  Rehearsal 
may  build  a  common  model  of  the  team  and  task  conditions  and  result  in  better  anticipation. 

The  present  study  focuses  on  team  planning  by  examining  the  relationship  between  planning  and 
teamwork  behaviors.  Unlike  Stout  et  al,  (1999),  the  findings  reported  here  are  based  on  the 
teamwork  behaviors  of  an  actual  and  intact  team  (i.e.,  squadron  staff)  operating  in  a  high  fidelity 
simulation  environment. 

2.  Method 

2.1  Participants 

Participants  were  14  active  duty  U.S.  Army  command  and  staff  officers  and  NCOs  at  squadron 
level.  They  participated  in  a  battle  command  reengineering  experiment  conducted  by  the 
Mounted  Maneuver  Battlespace  Lab  at  Fort  Knox,  KY,  in  April  1999.  Command  and  staff 
participants  were  equipped  with  futuristic  command,  control,  communication  and  computers  (C^) 
envisioned  for  a  2010+  timeframe.  As  depicted  in  Figure  1,  participants  were  organized  into 
four  nodes:  Command  1  and  2,  and  Control  1  and  2.  Command  1  and  2  included  the  battalion 
commander  and  deputy  commander,  respectively.  Control  1  and  2  alternated  between  assisting 
control  of  the  current  mission  and  planning  the  next  mission.  Each  node  had  an  officer  in 
charge,  friendly  operations  officer  or  NCO,  and  an  enemy  operations  officer  or  NCO.  The 
control  nodes  also  had  a  sensor  NCO,  responsible  for  operating  the  unmanned  aerial  vehicles 
attached  to  these  nodes. 


CONTROL  1 
Battle  Captain  (CPT) 
Friendly  OPs  (MSG) 
Enemy  OPs  (SEC) 
Sensor  NCO  (SEC) 


COMMAND  2 

Deputy  Commander  (MAJ) 

Ops  Officer  (CPT) 

OPS  NCO  (SEC) 


COMMAND  1 
Commander  (ETC) 
Effects  Ops  (CPT) 
Enemy  Ops  (CPT) 


CONTROL  2 
Battle  Captain  (CPT) 
Eriendly  OPs  (CPT) 
Enemy  OPs  (SEC) 
Sensor  NCO  (SEC) 


Figure  1.  Reengineered  battalion  command  group  structure. 


2.2  Measurement  Instruments 


An  Operations  Planning  Survey  (OPS)  was  developed  to  assess  the  training  designed  to  increase 
participant  knowledge  of  other  team  members'  roles  and  functions  during  tactical  operations. 

The  OPS  items  addressed  how  well  individual  command  and  staff  members  understood  the 
mission  and  team  members'  roles  and  information  requirements.  Items  included  statements  such 
as  "I  understood  my  role  in  the  mission"  and  "I  understood  the  information  requirements  of 
others."  Overall,  die  OPS  consisted  of  1 1  such  items.  Participants  responded  to  each  item  on  a 
five-point  scale,  ranging  from  "Strongly  Disagree"  to  "Strongly  Agree." 

A  Teamwork  Observation  Form  (TOF)  was  adapted  from  earlier  research  on  naval  command  and 
control  teams  (Entin  &  Serfaty  1999).  The  TOF  included  five  sections  on  teamwork  behaviors 
and  an  overall  effectiveness  rating.  The  behaviors  were:  communication;  monitoring;  back-up; 
coordination;  and  team  orientation.  A  sample  item  under  communication,  for  example,  read:  "To 
what  extent  were  errors  caused  by  inadequate  within  node  communication?"  Under  back-up 
behaviors,  a  sample  item  read:  "To  what  extent  did  node  members  anticipate  the  need  to  provide 
assistance  to  other  nodes?"  Overall,  the  TOF  consisted  of  14  such  items  with  two  to  four  items 
per  behavioral  category.  Observers  responded  to  each  TOF  item  on  a  seven-point  rating  scale 
with  behavioral  anchors  for  the  end  points. 

2.3  Procedure 

The  squadron  executed  one  operational  mission  per  day  during  the  six  days  of  evaluation.  For 
each  mission,  the  staff  received  an  operations  order  from  brigade  headquarters  and  planned  and 
executed  the  mission  in  coordination  with  supporting  company  level  commanders.  For  the  most 
part,  company-level  activities  were  performed  via  constructive  simulation,  as  directed  by  the 
company  commanders.  Observers  completed  the  TOF  twice  during  each  mission.  An  observer 
in  each  of  the  four  staff  nodes  rated  the  overall  node  observed.  Observers  rotated  among  the 
nodes  after  each  mission.  Participants  completed  the  OPS  at  the  end  of  each  mission. 

2.4  Analyses 

For  planning,  participants'  scores  were  based  on  only  five  of  the  eleven  OPS  scale  items.  These 
five  items  assessed  each  participant's  own  planning  versus  the  planning  performed  by  other 
members  of  the  participant's  node  or  other  nodes.  It  was  assumed  that  ratings  of  own 
performance  might  provide  more  accurate  and  reliable  data.  Total  scores  for  these  five  items 
were  averaged  over  all  members  in  each  node  for  each  mission.  For  teamwork  behaviors, 
observers'  TOF  ratings  on  each  behavioral  category  were  averaged  together  by  node  by  mission. 
Since  both  participant  and  observer  data  were  collected  on  each  of  the  six  missions,  a  total  of  24 
data  points  (four  nodes  x  six  missions)  for  both  planning  and  teamwork  behaviors  was  possible. 
Preliminary  review  of  this  data  based  on  scatter  plots  suggested  a  relationship  between 
participants'  perceptions  of  their  planning  and  observers'  ratings  of  teamwork. 

To  examine  these  relationships,  a  median  split  was  performed  on  the  data  for  the  planning  survey 
and  each  of  the  five  teamwork  behaviors.  Medians  were  approximate  because  of  the  distribution 
of  scores  and  the  desire  to  use  the  scale  midpoint  for  all  the  teamwork  behaviors.  Cross 
tabulations  were  generated  comparing  the  planning  scale  data  with  each  of  the  five  teamwork 
behaviors.  Since  these  data  were  not  independent  (i.e.,  each  of  the  nodes  accounted  for  six  data 
points  on  each  cross-tabulation),  inferential  statistics  were  not  used.  Note,  that  although  the 
following  Results  section  uses  terms  such  as  "higher"  and  "lower"  ratings  to  describe  the  data, 
these  terms  in  no  way  reflect  statistically  significant  differences  for  these  exploratory  findings. 


3.  Results 


Based  on  median  splits,  cross  tabulations  were  computed  between  planning  and  five  teamwork 
behaviors:  communication;  monitoring;  back-up;  coordination;  and  team  orientation.  All  five 
cross  tabulations  indicated  a  positive  relationship  between  planning  and  teamwork  behaviors. 
Overall,  participant  and  observer  ratings  ranged  from  71%-86%  concurrence  across  the  five 
cross  tabulations  examined. 

The  cross  tabulation  for  planning  and  communication  is  shown  in  Table  1.  Based  on  median- 
split  comparisons.  Table  1  indicates  that  participant  planning  and  observer  ratings  of 
communication  behaviors  concurred  77%  of  the  time  (17  of  22  rating  comparisons).  These 
concurring  ratings  were  evenly  split  on  the  above  (9)  and  below  (8)  median  ratings. 

Table  1.  Relation  of  Participant  Planning  Table  2.  Relation  of  Participant  Planning 

and  Observer  Communication  Ratings.  and  Observer  Monitoring  Ratings. 


Observer 

Communication  Ratings 

Participant 

Below 

At/ Above 

Planning  Ratings 

Midpoint 

Midpoint 

Below  Median 

N=8 

N=2 

%=80 

%=20 

Above  Median 

N=3 

N=9 

%=25 

%=75 

Observer 

Monitoring  Ratings 

Participant 

Below 

At/ Above 

Planning  Ratings 

Midpoint 

Midpoint 

Below  Median 

N=7 

N=2 

%=77 

%=23 

Above  Median 

N=3 

N=9 

%=25 

%=75 

Results  are  quite  similar  in  Table  2.  Assessments  of  participant  planning  and  observer 
monitoring  agreed  76%  of  the  time  (16  of  21  rating  comparisons).  The  agreement  of  the  ratings 
was  relatively  evenly  split  on  the  above  (9)  and  below  (7)  median  ratings. 

Tables  3  also  shows  good  agreement  between  participant  planning  and  observer  ratings  of  back¬ 
up  behavior.  Participant  ratings  of  planning  and  observer  ratings  of  backup  behavior  were 
consistent  81%  of  the  time  (17  of  21  rating  comparisons).  These  consistent  ratings  were  evenly 
split  on  above  (8)  and  below  (9)  median  ratings. 

Table  3.  Relation  of  Participant  Planning  Table  4.  Relation  of  Participant  Planning 

and  Observer  Backup  Ratings.  and  Observer  Coordination  Ratings. 


Observer 
Backup  Ratings 

Participant 

Below 

At/ Above 

Planning  Ratings 

Midpoint 

Midpoint 

Below  Median 

N=9 

N=1 

%=90 

%=10 

Above  Median 

N=3 

N=8 

%=27 

%=73 

Observer 

Coordination  Ratings 

Participant 

Below 

At/ Above 

Planning  Ratings 

Midpoint 

Midpoint 

Below  Median 

N=9 

N=1 

%=90 

%=10 

Above  Median 

N=2 

N=10 

%=17 

%=83 

Results  in  Table  4  are  quite  similar  to  Table  3.  Ratings  of  participant  planning  and  observer 
coordination  ratings  were  in  accordance  86%  of  the  time  (19  of  22  rating  comparisons).  These 
mutual  ratings  were  evenly  split  on  the  above  (10)  and  below  (9)  the  median. 


Table  5.  Relation  of  Participant  Planning 
and  Observer  Team  Orientation  Ratings. 


Observer  Team 
Orientation  Ratings 

Participant 

Below 

At/ Above 

Planning  Ratings 

Midpoint 

Midpoint 

Below  Median 

N=6 

N=3 

%=67 

%=33 

Above  Median 

N=3 

N=9 

%=25 

%=75 

Results  in  Table  5  shows  the  weakest  level  of 
agreement  between  participant  and  observer 
ratings.  Participant  ratings  of  planning  and 
observer  ratings  of  team  orientation  agreed 
71%  of  the  time  (15  of  21  rating  comparisons). 
These  ratings  were  somewhat  unevenly  split  on 
above  (9)  and  below  (6)  median  ratings. 

Table  6  allows  examination  of  observer  and 
participant  ratings  on  a  node-by-node  level  and 
by  Command  versus  Control  node  categories. 
Overall,  the  nodes  rated  most  favorably  on 
teamwork  behaviors  by  the  observers  were  the 
Command  nodes. 


Averaged  across  all  five  behaviors  and  the  two  node  categories,  observers'  Table  6  mean  (M) 
ratings  of  Command  node  teamwork  (M  =  5.3)  appear  markedly  more  positive  than  their  mean 
ratings  of  Control  node  teamwork  (M  =  2.5).  This  pattern  of  higher  Command  node  ratings  by 
observers  is  consistent  across  each  of  the  teamwork  behaviors  examined.  In  contrast,  participant 
ratings  of  their  own  planning  performance  disclose  a  notable  exception  to  a  pattern  of  higher 
ratings  for  Command  nodes.  While  Control  2  participants  rated  their  planning  lower  than  any 
other  node.  Control  1  participants  rated  their  planning  higher  than  any  other  node.  Notably,  this 
disconnect  between  observer  and  participant  ratings  for  the  Control  1  node  account  for  many  of 
the  rating  disagreements  previously  identified  in  the  cross  tabulation  results,  particularly  the 
tabulations  shown  in  the  lower-left  cells  of  Tables  1-5  that  indicate  higher  ratings  of  planning 
and  lower  ratings  of  teamwork. 

Table  6.  Mean  Observer  Teamwork  and  Participant  Planning  Ratings  by  Node 


Node 

Observer 

Participant 

Commo 

Monitor 

Backup 

Coord 

Team 

Orient 

Planning 

Command  1 

5.28 

5.10 

4.47 

4.80 

4.88 

4.29 

Command  2 

5.54 

5.42 

5.00 

5.46 

5.33 

4.20 

Control  1 

2.90 

2.50 

2.20 

2.56 

2.71 

4.31 

Control  2 

2.38 

2.25 

1.94 

2.54 

3.15 

3.85 

Note.  Commo  =  Communication;  Monitor  =  Performance  Monitoring;  Coord  =  Coordination;  and  Team  Orient  = 
Team  Orientation.  Scale  midpoints  were  4.00  on  observer  ratings  and  3.00  on  participant  ratings. 

4.  Discussion  and  Conclusions 

Overall,  the  findings  generally  indicate  a  consistent  and  positive  relationship  between  team 
planning  and  teamwork.  Participant  ratings  of  their  planning  concurred  with  observer  ratings  of 
their  teamwork  behaviors  between  71%-86%  on  the  five  behavioral  categories  examined.  When 
participants  perceived  that  there  was  better  planning  for  a  mission,  observers  generally  assessed 
teamwork  behaviors  during  that  mission  as  better.  This  suggests  a  relationship  between  efficacy 
of  mission  planning  and  the  degree  of  teamwork  observed.  Such  a  relationship  is  consistent  with 


the  literature  on  team  anticipation  cited  earlier.  That  is,  planning  would  provide  team  members 
with  knowledge  of  what  information  or  actions  other  team  members  might  need,  and  under  what 
conditions  team  members  might  need  it,  during  the  upcoming  mission.  This  knowledge  would 
allow  team  members  to  provide  the  information  or  action  before  it  was  requested,  thus  enhancing 
team  performance. 

However,  potential  differences  in  Command  versus  Control  node  ratings  bear  consideration. 
Higher  participant  and  observer  ratings  were  generally  associated  with  the  Command  versus 
Control  nodes,  with  the  exception  of  participants'  self-assessment  of  their  planning  in  Control  1. 
Notably,  Command  1  and  2  had  the  most  senior  and  experienced  personnel.  The  battalion 
commander,  a  Lieutenant  Colonel,  was  in  charge  of  Command  1,  his  enemy  operations  officer 
was  the  intelligence  officer  (S2)  for  the  squadron,  and  his  friendly  operations  officer  was  the 
squadron  operations  officer  (S3).  The  squadron’s  Deputy  Commander,  a  Major,  was  in  charge 
of  Command  2.  Not  only  were  these  aforementioned  officers  some  of  the  most  experienced  in 
the  squadron,  but  as  with  nearly  all  units,  duty  position  fills  were  hand-picked  by  the 
commander.  In  contrast,  less  senior  and  experienced  officers,  none  exceeding  the  rank  of 
Captain,  served  in  Control  1  and  2. 

Observer  rating  differences  between  Command  and  Control  nodes  might  accurately  reflect 
behavioral  differences  or  a  biasing  expectation  that  more  experienced  team  members  exhibit 
better  teamwork  skills.  Without  more  objective  measures  of  participant  and  node  performance 
(which  were  not  available  from  this  assessment),  we  cannot  be  certain  whether  the  observers 
provided  accurate  measures  of  teamwork  or  whether  the  ratings  were  affected  by  the  rank  and 
position  of  the  participants.  One  result  that  counters  a  biased  observer  explanation  is  the  fact  that 
on  each  of  the  five  behavioral  categories  examined,  the  Deputy  Commander's  node.  Command  2, 
received  higher  average  ratings  from  observers  than  the  Commander's,  Command  1.  This  result 
suggests  that  something  other  than  rank  and  position  bias  on  the  observers’  part  was  occurring. 

In  addition,  the  rotation  of  observers  across  nodes  reduces  observer  bias  interpretations. 

One  finding  of  special  interest  may  be  the  comparatively  large  discrepancy  between  observer  and 
participant  ratings  for  the  Control  1  node.  As  noted,  observers  consistently  rated  the  teamwork 
behaviors  of  Control  participants  below  the  Command  node  participants.  And  the  relatively  low 
ratings  that  Control  2  participants  provided  on  their  planning  performance  appeared  to  concur 
with  observer  ratings.  However,  Control  1  participants'  rated  their  planning  performance  higher 
than  participants  in  any  other  node.  Control  I's  relatively  high  self-ratings  contrasted  sharply 
with  observer  ratings  of  their  teamwork  behaviors.  A  plausible  interpretation  is  that  less 
experienced  and/or  novice  personnel  are  less  able  or  likely  to  accurately  assess  their  own 
performance,  particularly  more  cognitive  performance  such  as  planning.  If  the  self-ratings  by 
Control  1  participants  were  inaccurate,  as  seemingly  indicated  by  observer  ratings  of  their 
behavior,  more  explicit  feedback  on  planning  and  teamwork  may  have  been  needed.  From  a 
training  perspective,  this  interpretation  reinforces  the  need  for  extrinsic  performance  assessment 
and  feedback,  particularly  during  the  earlier  phases  of  training. 

Overall,  the  results  are  promising  in  that  they  were  obtained  with  an  actual  Army  squadron 
commander  and  staff  in  a  high  fidelity  simulation.  Although  exploratory,  the  findings  suggest 
that  planning  and  teamwork  are  closely  related  especially  for  a  reengineered  command  and 
control  organization,  as  examined.  One  area  for  future  research  is  to  experimentally  manipulate 
the  amount  of  planning  performed  by  nodes  or  staffs,  and  examine  its  effect  on  teamwork 
behaviors.  Another  area  is  the  need  for  more  objective  assessments  of  staff  performance. 
Instrumentation  of  the  C"^  systems  that  will  be  used  by  future  command  and  staff  organizations 
might  help  provide  more  objective  and  precise  automated  measures  of  staff  performance  (Throne 
et  al.,  1999).  In  addition,  automated  measures  might  improve  performance  assessment  and 
feedback,  particularly  during  the  earlier  phases  of  training  when  trainees  may  not  be  able  to 
accurately  assess  their  own  performance.  Certainly,  research  with  more  and  diverse  teams  is 


necessary  to  determine  more  precisely  if  and  how  better  planning  is  associated  with  better 

teamwork. 
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